Calls for transparency in AI systems are growing in number and urgency from diverse stakeholders ranging from regulators to researchers to users (with a comparative absence of companies developing AI). Notions of transparency for AI abound, each addressing distinct interests and concerns. In computer security, transparency is likewise regarded as a key concept. The security community has for decades pushed back against so-called security by obscurity -- the idea that hiding how a system works protects it from attack -- against significant pressure from industry and other stakeholders. Over the decades, in a community process that is imperfect and ongoing, security researchers and practitioners have gradually built up some norms and practices around how to balance transparency interests with possible negative side effects. This paper asks: What insights can the AI community take from the security community's experience with transparency? We identify three key themes in the security community's perspective on the benefits of transparency and their approach to balancing transparency against countervailing interests. For each, we investigate parallels and insights relevant to transparency in AI. We then provide a case study discussion on how transparency has shaped the research subfield of anonymization. Finally, shifting our focus from similarities to differences, we highlight key transparency issues where modern AI systems present challenges different from other kinds of security-critical systems, raising interesting open questions for the security and AI communities alike.

At the RSA security conference in San Francisco this week, there's been a feeling of inevitability in the air. At talks and panels across the sprawling Moscone convention center, at every vendor booth on the show floor, and in casual conversations in the halls, you just know that someone is going to bring up generative AI and its potential impacts on digital security and malicious hacking. NSA cybersecurity director Rob Joyce has been feeling it, too. "You can't walk around RSA without talking about AI and malware," he said on Wednesday afternoon during his now annual "State of the Hack" presentation. "I think we've all seen the explosion. I won't say it's delivered yet, but this truly is some game-changing technology."

New Delhi, NFAPost: Sophos, a global leader in innovating and delivering cybersecurity as a service, released new research on how the cybersecurity industry can leverage GPT-3, the language model behind the now well-known ChatGPT framework, as a co-pilot to help defeat attackers. The latest report, "GPT for You and Me: Applying AI Language Processing to Cyber Defenses," details projects developed by Sophos X-Ops using GPT-3's large language models to simplify the search for malicious activity in datasets from security software, more accurately filter spam, and speed up analysis of "living off the land" binary (LOLBin) attacks. Sophos Principal Threat Researcher Sean Gallagher said Since OpenAI unveiled ChatGPT back in November, the security community has largely focused on the potential risks this new technology could bring. "Can the AI help wannabee attackers write malware or help cybercriminals write much more convincing phishing emails? Perhaps, but, at Sophos, we've long seen AI as an ally rather than an enemy for defenders, making it a cornerstone technology for Sophos, and GPT-3 is no different. The security community should be paying attention not just to the potential risks, but the potential opportunities GPT-3 brings," said Sophos Principal Threat Researcher Sean Gallagher.

Beneath the buzz, the metaverse is arriving in both predictable and unexpected ways. Some new experiences using headsets and mixed reality will be in your face – quite literally – but other implications will be harder to spot. As with all new categories, we'll see intended and unintended innovations and experiences, and the security stakes will be higher than we imagine at first. There is an inherent social engineering advantage with the novelty of any new technology. In the metaverse, fraud and phishing attacks targeting your identity could come from a familiar face – literally – like an avatar who impersonates your coworker, instead of a misleading domain name or email address.

When it comes to detecting bias in algorithms, researchers are trying to learn from the information security field – and particularly, from the bug bounty-hunting hackers who comb through software code to identify potential security vulnerabilities. The parallels between the work of these security researchers and the hunt for possible flaws in AI models, in fact, is at the heart of the work carried out by Deborah Raji, a research fellow in algorithmic harms for the Mozilla Foundation. Presenting the research she has been carrying out with advocacy group the Algorithmic Justice League (AJL) during the annual Mozilla Festival, Raji explained how along with her team, she has been studying bug bounty programs to see how they could be applied to the detection of a different type of nuisance: algorithmic bias. SEE: An IT pro's guide to robotic process automation (free PDF) (TechRepublic) Bug bounties, which reward hackers for discovering vulnerabilities in software code before malicious actors exploit them, have become an integral part of the information security field. Major companies such as Google, Facebook or Microsoft now all run bug bounty programs; the number of these hackers is multiplying, and so are the financial rewards that corporations are ready to pay to fix software problems before malicious hackers find them.

Machine learning (ML) is making incredible transformations in critical areas such as finance, healthcare, and defense, impacting nearly every aspect of our lives. Many businesses, eager to capitalize on advancements in ML, have not scrutinized the security of their ML systems. Today, along with MITRE, and contributions from 11 organizations including IBM, NVIDIA, Bosch, Microsoft is releasing the Adversarial ML Threat Matrix, an industry-focused open framework, to empower security analysts to detect, respond to, and remediate threats against ML systems. During the last four years, Microsoft has seen a notable increase in attacks on commercial ML systems. Market reports are also bringing attention to this problem: Gartner's Top 10 Strategic Technology Trends for 2020, published in October 2019, predicts that "Through 2022, 30% of all AI cyberattacks will leverage training-data poisoning, AI model theft, or adversarial samples to attack AI-powered systems."

The term Artificial Intelligence (AI) was coined by John McCarthy in 1956. AI is defined in the Oxford English Dictionary as "the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages." Artificial Intelligence (AI) is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans. AI is increasingly being used in the defence sector to boost the military capabilities in many developing nations of the world. In a December 2019 strategic research paper entitled, "A Candle in the Dark: US National Security Strategy for Artificial Intelligence", Stephen Rodriguez and Tate Nurkin shed more light on this aspect.

The security community has found an important application for machine learning (ML) in its ongoing fight against cybercriminals. Per Recorded Future, many of us are turning to ML-powered security solutions like Lastline that analyze network traffic for anomalous and suspicious activity. In turn, these ML solutions defend us from digital threats better than other solutions can by drawing on their evolving knowledge of what a network attack looks like. Digital attackers are aware of the fact that security solutions are using ML for security purposes. They also know that there are certain limitations when it comes to applying artificial intelligence to computer security. This explains why digital criminals are leveraging ML to their advantage in something known as "adversarial machine learning."

Machine learning is still in the hype stage as actually leveraging the technology is still a challenging feat, especially where data wrangling is concerned, delegates of Splunk's .conf During the event's opening keynote, Richard Campione, chief product officer at Splunk, noted that machine learning is getting a lot of buzz these days, but still has room to grow when it comes to adoption. "Why don't we see it everywhere? Why is it mainly hype right now? Because it's difficult to do in practice. This is the challenge we're taking on," he said.

The rise of behavioral analytics, machine learning, artificial intelligence, or whatever the latest nomenclature is currently being promoted by vendors, has taken the security community by storm and showing no signs of stopping. It's almost impossible not to see these phrases mentioned on new preventative solutions going to market and rightfully so. With an industry accustomed to relying on static signatures, known bad hashes and singular alerting, this technology is a welcome relief for defenders and we've seen the market capitalize on our desire for it. The progression of the security industry towards technologies that welcome behavior analysis over static alerting is a step forward in the evolution of detection and defense. These solutions aren't perfect by any means, but the progressive mindset in closing the detection gap is a step forward.